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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 



Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 

earned patent term adjustment. See 37 CFR 1.704(b). 



3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1 - 24 is/are pending in the application. 

4a) Of the above claim(s) 21 - 23 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) IEl Claim(s) 1 -3.5. 9. 11 - 13. 15-20. 24 is/are rejected. 

7) |El Claim(s) 4. 6-8. 10. 14 is/are objected to. 

8) D Claim(s) ; are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)Q None of: 

1 .Q Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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1)^ Responsive to communication(s) filed on 10 February 2005 . 
2a)^ This action is FINAL. 2b)D This action is non-final. 
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1 . The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

2. Claims 1 , 5, 9, 1 1 - 1 3, 1 5 - 20, 24 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Liu ("Liu"; US #5,953,005 A) in view of Fujita et al. ("Fujita"; US 
#6,600,874 B1). 

As per independent claim 1's "synchronizing visual information with audio 
playback" (see also independent claim 24), Liu teaches that MULTIMEDIA ACCESS 
can be obtained via a Karaoke application.. .where the user desires to access songs 
which are most popular at a given time (Abstract ;col 2, lines 34 - 52). In the Javaoke 
arrangement taught by Liu (see figs 2, 4), "receiving a user selection of a desired audio 
file" (as from claim 24's "stored listing"), followed by "initiating play of the desired audio 
file on an audio playback device" takes place — the chosen song plays at box 134 , fig 7 
(col 6, lines 28 - 40). Since the Javaoke Applet runs at a display browser such as in fig 
2 (and therefore is disclosed as capable of "displaying visual information associated 
with the desired audio file" in the form of lyrics 102 , fig 4), "the commencement of 
playing the desired audio file by the audio playback device and the commencement of 
the displaying step by the display device are a function of a signal from the display 
device". In the case of Liu, since separate processing is inherently provided for the 
visual and audio portions of the presentation, the "device" components are such that the 
"display device" is "separate from said audio playback device", internal to the overall 
machine that runs the Javaoke Applet . 
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Liu does not enter in such specifics of Karaoke as those that would explicitly 
read upon the claimed use of "timestamp data such that the visual information is 
displayed synchronously with the playing of the desired audio file", though such an 
application has at least an implication that some form of matching lyrics to position in 
the song duration is required. 

However, Fujita, in DETECTING STARTING AND ENDING POINTS OF SOUND 
SEGMENT IN VIDEO by obtaining a thresholded envelope of a sound signal waveform 
(Abstract) must make note of time point data, as described at col 14, line 53 - col 15, 
line 45. A time point as per Fujita reads upon a reasonably broad interpretation of the 
claimed "timestamp", since both are used to mark a position in time that is pertinent to a 
later process. 

Therefore, it would'have been obvious to a person having ordinary skill in the art 
at the time of applicant's invention to use a "timestamp data" collection as per Fujita, in 
conjunction with the Liu Javaoke arrangement , where control from the "display device" 
reads out a "desired audio file" along with "visual information", the motivation being to 
arrive at the correct timing for lyrics in the Liu Karaoke application . 

As per claim 5's "analyzing the audio file" in order to produce such "timestamp 
data", this is a central focus of Fujita, as noted above — please note the involved . 
processing to generate an envelope, as in Fujita's fig 3. Encoded within the timing of 
the audio will be "tempo information extracted from music" (claim 9). 

As seen in Liu's fig 2 (see also col 4, lines 13 - 33), "the visual information 
comprises text lines" (claim 11), and also the "title of the desired audio file" (claim 12). 
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As in claim 13, the song choices in Liu are "from a list stored in a memory 
associated with the display device": the host servers as seen in fig 1 are disclosed as 
being so "associated". When the Javaoke applet is invoked in Liu, "a signal" is sent 
"from the display device to the audio playback device to cause the audio playback 
device to start" (claim 1 5), when the components of Liu are seen with the division into 
supervisory visual and accessory audio that falls within reasonable interpretation. 

Independent claim 16 chooses to concentrate on the audio analysis aspect of the 
present invention, as it relates to a "segment" that "may be displayed synchronously 
with playing of the corresponding segment in the audio data" in the style suggested by 
Liu, where "visual information consisting of text corresponding to speech" is produced 
by the Javaoke arrangement. Specifically, an "acoustic feature" and "pauses" are found 
"within the audio data", for the purpose of "segmenting the audio data" followed by 
"generating at least one timestamp value". But such analysis, as noted above, is also a 
central feature in Fujita, where a segmented representation such as that in window 315 , 
fig 3 (see also fig 4) is made from the input waveform. 

When "the at least one segment" as found in Fujita is used to direct the lyrics of 
Liu, the "segment refers to lyrics of a song" (claim 17), and such "differentiation", 
according to the song's author's intentions, will be between those for "one of the male 
gender and the female gender" (claim 18). 

As per claim 19's "indication of a tempo of the visual information", such 
information will be imparted to the Liu display, on the basis of the underlying "tempo" in 
the Fujita input audio. Fujita's analysis, as noted above, produces a thresholded signal 



Serial Number: 09/998,033 Page 5 

Art Unit: 2173 

that "separates the audio data into voice segments and non-voice segments" (claim 20), 
when "non-voice segments" takes the reasonably broad interpretation of low audio level 
or silence. 

3. Claim 2 is rejected under 35 U.S.C. 103(a) as being unpatentable over Liu in 
view of Fujita and Sai et al. ("Sai"; US #5,691 ,494 A). 

The Liu connection from "display device" to the "audio playback device" is not 
explicitly shown as being "an infrared signal", due to the close coupling of those two 
devices in the instance given. However, Sai's highly analogous technique for 
PROVIDING KARAOKE SERVICE uses a commander 5 of an infrared remote control 
type at the Karaoke Box location (col 3, lines 37 - 50). The source of commands in Sai 
is thus infrared -connected to the "audio playback device". 

It would have been further obvious to the person having ordinary skill in the art at 
the time of applicant's invention to implement the Javaoke control as per Liu/Fujita via 
an "infrared signal", with the motivation being to avoid unnecessarily cumbersome 
wiring, when the components are separated by significant physical distance. 

4. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Liu in 
view of Fujita and Ng et al. ("Ng"; US #5,648,628). 

Liu, while having a "display device" that is capable of supplying "a signal", is not 
explicitly disclosed as using "a handheld device". However, Ng's KARAOKE DEVICE , 
with an LCD screen for displaying the title and lyrics of the song (Abstract), uses a 
console as illustrated in fig 3, in a portable configuration (col 2, lines 37 - 39). 
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It would thus have been finally obvious to the person of ordinary skill in the art to 
use Ng's portable (and thus, "handheld") "display device" to achieve the "timestamp"- 
oriented "displaying" of Fujita's modification to Liu, with the motivation being to enable a 
less-restricted placement of users within a given area. 

5. Applicant's arguments filed 10 February 2005 have been fully considered but 
they are not persuasive. 

At page 12 of the remarks, applicant argues that since "Liu refers to integrated 
Karaoke or computer systems wherein the display device and the audio playback 
device are integrated with one another", "there is no need for the display device to send 
a signal to the audio playback device to begin playing back the audio data". However, it 
is enough that Liu teaches the use of a "display portion" and an "audio playback portion" 
in the overall Javaoke system, since these are recognizably "separate" portions. In this 
situation, when one views Liu as having a visual and central processing capacity that 
generates the visual portion, it will have to send control signals to effect a proper audio 
playback. 

Concerning claim 16, applicant next argues (page 13) that "neither Liu nor Fujita 
et al. disclose or suggest how the detected information in Fujita et al. may be applied to 
the text forming the visual information". However, all that claim 16 recites is "visual 
information consisting of text corresponding to speech occurring in the audio data", 
which reads directly upon the lyrics presented in Liu, which are coordinated with "the 
playing back of the corresponding at least one segment of the audio data", when the "at 
least one segment" is found as per Fujita. 
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As per claim 2, applicant argues (page 14) that "[w]hile Sai et al. discloses 
controlling the Karaoke apparatus with an infrared remote control, this is typical of 
virtually all consumer electronic audio/video devices", with the claimed invention instead 
directed to a "display device" that "sends a (infrared) signal to a separate audio 
playback device for initiating playback of the audio data". However, Sai remains 
relevant, for showing that the control portion of a Karaoke system can connect by I R to 
that portion which provides the audio. In the event of Liu's separation of such units, it 
would thus have been an obvious mode of control, for the reasons stated above. 

As per claim 3, applicant argues (page 14) that "while the Ng et al. device may 
arguably be handheld, the Ng et al. device is a complete system", so that "there is no 
need for, and no disclosure or suggestion of, a display device sending a signal to a 
separate audio playback device". However, this feature is the reason the Examiner 
relies upon Liu. Ng is additionally cited to show a handheld implementation of the 
control unit. 

6. Claims 4, 6 - 8, 10, 14 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

The reasons for allowance are those given in the previous office action, mailed 6 
December 2004. 
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7. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

8. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Raymond J. Bayerl whose telephone number is (571) 
272-4045. The examiner can normally be reached on M - Th from 9:00 AM to 4:00 PM 
ET. 

9. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca, can be reached on (571 ) 272-4048. All patent application 
related correspondence transmitted by FAX must be directed to the central FAX 
number (703) 872-9306. 

1 0. Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (571) 272- 



2100. 
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